Method and apparatus with speech processing

ABSTRACT

Disclosed is a method and apparatus for processing a speech. The method includes obtaining context information from a speech signal of a user using a neural network-based encoder, determining intent information of the speech signal based on the context information, determining, based on the context information, attention information corresponding to a segment included in the speech signal, and determining, based on the attention information, a segment value of the segment by recognizing, using a decoder, a portion of the context information identified as corresponding to the segment.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit under 35 USC § 119(a) of KoreanPatent Application No. 10-2019-0125597 filed on Oct. 10, 2019 in theKorean Intellectual Property Office, the entire disclosure of which isincorporated herein by reference for all purposes.

BACKGROUND 1. Field

The following description relates to a method and apparatus with speechprocessing.

2. Description of Related Art

Speech recognition technology may be used to recognize a user's speech.The recognized speech of the user may be converted into text throughspeech recognition. Natural language understanding (NLU) is a technologyfor allowing a machine to understand one or more languages, e.g., in aconversational context. With the development of speech recognition andNLU technologies, users have a growing need for voice commands tocontrol devices or act as agents.

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

In one general aspect, a speech processing method includes obtainingcontext information from a speech signal of a user using a neuralnetwork-based encoder, determining intent information of the speechsignal based on the context information, determining, based on thecontext information, attention information corresponding to a segmentincluded in the speech signal, and determining, based on the attentioninformation, a segment value of the segment by recognizing, using adecoder, a portion of the context information identified ascorresponding to the segment.

In response to a determination that the speech signal may include aplurality of segments, determining segment values of the segments byrecognizing, in parallel using a plurality of decoders, portions of thecontext information identified as corresponding to the respectivesegments based on attention information respectively corresponding tothe plurality of segments.

The speech processing method may further include determining typeinformation of the segment included in the speech signal based on thecontext information. The determining of the segment value of the segmentmay further include recognizing the identified portion based on the typeinformation of the segment.

The determining of the segment value of the segment may further includerecognizing the identified portion based on the intent information ofthe speech signal.

Type information of the segment may be determined based on the contextinformation and the intent information of the speech signal.

The intent information may be further determined based on typeinformation of one or more segments included in the speech signal.

A plurality of segments included in the speech signal may besequentially identified by a segment classifier provided in the form ofa decoder.

The determining of the segment value of the segment may includesequentially determining segment values of the respective segments inresponse to the segments being sequentially identified by the segmentclassifier provided in the form of a decoder.

The speech processing method may further include performing an operationcorresponding to the intent information based on the segment value ofthe segment and the type information.

A non-transitory computer-readable storage medium may store instructionsthat, when executed by a processor, cause the processor to perform thespeech processing method above.

In another general aspect, a speech processing apparatus includes one ormore processors. The one or more processors may be configured to obtaincontext information from a speech signal of a user using a neuralnetwork-based encoder, determine intent information of the speech signalbased on the context information, determine, based on the contextinformation, attention information corresponding to a segment includedin the speech signal, and determine, based on the attention information,a segment value of the segment by recognizing, using a decoder, aportion of the context information identified as corresponding to thesegment.

In response to a determination that the speech signal includes aplurality of segments, the one or more processors may be configured todetermine segment values of the segments by recognizing, in parallelusing a plurality of decoders, portions of the context informationidentified as corresponding to the respective segments based onattention information respectively corresponding to the plurality ofsegments.

The one or more processors may be further configured to determine typeinformation of the segment included in the speech signal based on thecontext information, and determine, based on the type information of thesegment, the segment value of the segment by recognizing the identifiedportion.

The one or more processors may be further configured to determine, basedon the intent information of the speech signal, the segment value of thesegment by recognizing the identified portion.

Type information of the segment may be determined based on the contextinformation and the intent information of the speech signal.

The one or more processors are further configured to determine theintent information of the speech signal based on type information of oneor more segments included in the speech signal.

A plurality of segments included in the speech signal may besequentially identified by a segment classifier provided in the form ofa decoder.

The one or more processors may be further configured to sequentiallydetermine segment values of the respective segments in response to thesegments being sequentially identified by the segment classifierprovided in the form of a decoder.

The one or more processors may be further configured to perform anoperation corresponding to the intent information based on the segmentvalue of the segment and the type information.

In another general aspect, a speech processing method includesdetermining context information from a speech signal of a user,determining, based on the context information, intent information andsegments of the speech signal, determining attention informationcorresponding to the segments excluding the intent information,determining, based on the attention information, segment values of thesegments, and executing the intent information for the segment values.

The segment values may be determined in parallel using a plurality ofdecoders.

The segment values may be determined based on the intent information.

Each of type information of the segments may be determined based on thecontext information and the intent information.

The intent information may be further determined based on typeinformation of one or more of the segments.

Other features and aspects will be apparent from the following detaileddescription, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of processing a natural language based ona speech processing apparatus.

FIGS. 2 and 3 illustrate examples of operations of speech processingapparatuses.

FIG. 4 illustrates an example of a speech processing method.

FIG. 5 illustrates an example of a speech processing apparatus.

Throughout the drawings and the detailed description, unless otherwisedescribed or provided, the same drawing reference numerals will beunderstood to refer to the same elements, features, and structures. Thedrawings may not be to scale, and the relative size, proportions, anddepiction of elements in the drawings may be exaggerated for clarity,illustration, and convenience.

DETAILED DESCRIPTION

The following detailed description is provided to assist the reader ingaining a comprehensive understanding of the methods, apparatuses,and/or systems described herein. However, various changes,modifications, and equivalents of the methods, apparatuses, and/orsystems described herein will be apparent after an understanding of thedisclosure of this application. For example, the sequences of operationsdescribed herein are merely examples, and are not limited to those setforth herein, but may be changed as will be apparent after anunderstanding of the disclosure of this application, with the exceptionof operations necessarily occurring in a certain order. Also,descriptions of features that are known in the art may be omitted forincreased clarity and conciseness.

The features described herein may be embodied in different forms, andare not to be construed as being limited to the examples describedherein. Rather, the examples described herein have been provided merelyto illustrate some of the many possible ways of implementing themethods, apparatuses, and/or systems described herein that will beapparent after an understanding of the disclosure of this application.

Terms, such as first, second, and the like, may be used herein todescribe components. Each of these terminologies is not used to definean essence, order or sequence of a corresponding component but usedmerely to distinguish the corresponding component from othercomponent(s). For example, a first component may be referred to as asecond component, and similarly the second component may also bereferred to as the first component.

It should be noted that if it is described that one component is“connected”, “coupled”, or “joined” to another component, a thirdcomponent may be “connected”, “coupled”, and “joined” between the firstand second components, although the first component may be directlyconnected, coupled, or joined to the second component.

The singular forms “a”, “an”, and “the” are intended to include theplural forms as well, unless the context clearly indicates otherwise. Itwill be further understood that the terms “comprises/comprising” and/or“includes/including” when used herein, specify the presence of statedfeatures, integers, steps, operations, elements, and/or components, butdo not preclude the presence or addition of one or more other features,integers, steps, operations, elements, components and/or groups thereof.

Unless otherwise defined, all terms, including technical and scientificterms, used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which this disclosure pertains and afteran understanding of this disclosure of this application. Terms, such asthose defined in commonly used dictionaries, are to be interpreted ashaving a meaning that is consistent with their meaning in the context ofthe relevant art and the disclosure of this application, and are not tobe interpreted in an idealized or overly formal sense unless expresslyso defined herein.

Hereinafter, examples are described in detail with reference to theaccompanying drawings. The following specific structural or functionaldescriptions are exemplary to merely describe the examples, and thescope of the examples is not limited to the descriptions provided in thepresent specification. Like reference numerals in the drawings denotelike elements, and a known function or configuration will be omittedherein.

FIG. 1 illustrates an example of processing a natural language based ona speech processing apparatus.

FIG. 1 depicts an example of a controlling operation of an electronicdevice 120 based on a user's determined intent, e.g., identified by aspeech processing apparatus 110 from the user's captured speech signal.

The speech processing apparatus 110 determines segment (or slot) in aspeech signal. The segment may include a segment name and segment value.The segment name included in the speech signal may include intentinformation, recipient information, and content information, forexample. The intent information may indicate the user's intention fromthe speech signal uttered by the user. Here, the intent information mayrefer to an operation that the user desires to perform through anutterance, and may include various operations that may be performed bythe electronic device 120. The operations of the intent information thatmay be performed may include sending a message, checking a receivedmessage, making a call, checking/creating/editing/deleting a schedule,playing a song/video, ordering goods in online market, and the like. Inan example, the segment value may include a phrase or word correspondingto a keyword included in the speech signal to perform the operationcorresponding to the user intent. Herein, it is noted that use of theterm ‘may’ with respect to an example or embodiment, e.g., as to what anexample or embodiment may include or implement, means that at least oneexample or embodiment exists where such a feature is included orimplemented while all examples and embodiments are not limited thereto.

In the example of sending a text message depicted in FIG. 1, “Peter”corresponds to a recipient information and “Have you had lunch”corresponds to the content information are segments. In other words,when a voice command of the user is regarded as a command for executionof a function in a program, the intent information is the name of afunction to be executed, and the segment value is the parameter passedto the function.

The speech processing apparatus 110 uses an encoder-decoder model todetermine the intent information of the speech signal and the segment.The encoder-decoder model is a neural network including a plurality oflayers and includes an encoder and a decoder. The encoder may determinecontext information with respect to the provided speech signal of theuser. The context information may be a vector sequence, which may be asummary information containing the meaning of the speech signal. Ingeneral, a decoder determines a sentence sequence corresponding to aspeech recognition result based on context information. Words may besequentially generated to determine the sentence sequence. Since, in anexample, a word generated in a current inference or run of the decoderis input again into the decoder to generate a word of a subsequentinference or run of the decoder, the time for performing speechrecognition increases, and the recognition accuracy may decrease. Thus,when speech recognition is partially performed only with respect to akeyword for performing the operation corresponding to the user intent inthe speech signal, the time for performing speech recognition decreases,and the recognition accuracy improves.

To achieve the foregoing, an attention mechanism may be applied to theencoder-decoder model. For example, a portion corresponding to a segmentin the context information is identified based on attention informationcorresponding to the segment, and the segment value of the segment isdetermined by recognizing only the identified portion using the decoder.The attention information is information indicating a portion of contextinformation desiring of attention for the decoder to output a segmentvalue of the segment as a recognition result.

For ease of description, in an example, it is assumed that a userutters: “Can you text Peter have you had lunch?,” as in the example ofFIG. 1. The speech processing apparatus 110 determines intentinformation of sending a message from the speech signal corresponding tothe utterance of the user, and determines segments “recipient: Peter”and “content: have you had lunch.” The electronic device 120 performs anoperation corresponding to the information determined by the speechprocessing apparatus 110. For example, the electronic device 120 sends atext message: “have you had lunch” to Peter.

The electronic device 120 may be a device that the user desires tocontrol, and may include in or be representative of, various computingdevices such as a smart phone, a tablet computer, a laptop computer anda personal computer, various wearable device such as a smart watch and asmart eyeglass, various home appliances such as a smart speaker, a smartTV, and a smart refrigerator, a smart vehicle, and a smart kiosk, asnon-limiting examples. Accordingly, through the speech processingapparatus 110, an artificial intelligence (AI) assistant configured tocontrol the electronic device 120 with a voice command of the user maybe implemented.

In FIG. 1, an example in which the speech processing apparatus 110 andthe electronic device 120 are provided as separate devices andcommunicate with each other through wired and/or wireless networks arealso illustrated. In this example, the speech signal of the user may betransmitted to the speech processing apparatus 110, for example, aremote server and information determined by the speech processingapparatus 110 transmitted to the electronic device 120, such that theelectronic device 120 performs a predetermined operation. However, theabove example is provided only for ease of description, and examples arenot limited thereto. The description herein also similarly applies to anexample in which the speech processing apparatus 110 and the electronicdevice 120 are implemented as a single device.

Hereinafter, the operation of the speech processing apparatus 110 willbe described in further detail.

FIGS. 2 and 3 illustrate examples of operations of speech processingapparatuses.

Referring to FIG. 2, a speech processing apparatus includes an encoder210, an intent classifier 220, a segment classifier 230, and speechrecognition decoders 241 and 242, for example.

The encoder 210 may output context information as an encoding resultfrom a captured or stored speech signal of a user. In this example, theencoder 210 may output the context information by receiving the entirespeech signal in response to completion of an utterance of the user oroutput corresponding context information in real-time each time a speechsignal is partially received during the utterance of the user.

The intent classifier 220 may determine intent information of the speechsignal based on the context information. For example, the intentclassifier 220 may be a neural network configured to determine theintent information of the speech signal from the context information.The segment classifier 230 may be a classifier provided in the form ofsoftmax layer(s) or functions, for example.

The segment classifier 230 may identify one or more segments included inthe speech signal based on the context information. The segmentclassifier 230 may determine the type information of a segment includedin the speech signal based on the context information. For example, thesegment classifier 230 may be a classifier provided in the form of asigmoid capable of multi-selection. The type information is informationindicating the meaning of a segment value of the segment, and mayinclude, for example, a recipient or content in a case of sending amessage as shown in FIG. 1.

Further, the segment classifier 230 may determine attention informationcorresponding to the segment included in the speech signal based on thecontext information. If the speech signal includes a plurality ofsegments, the segment classifier 230 may determine attention informationcorresponding to each segment.

In an example, the intent classifier 220 may determine the intentinformation of the speech signal based on both the context informationfrom the encoder 210 and the type information of the segment determinedby the segment classifier 230. For example, in response to the segmentclassifier 230 determining type information of segments corresponding toa recipient, message content, and a sending time, for example, theintent information of “sending a message” may be determined relativelyquickly with relatively high accuracy further based on the typeinformation of the segments.

In general, it is a verb in a sentence uttered by the user that may bestindicate an intent of the user, with segments being determinable fromthe remaining constituents of the sentence. For example, in a languagewhere a verb may be included or represented at the end of a sentence,like Korean, a user's intent is included or represented in the rearportion of an utterance. Thus, in an example of obtaining contextinformation through real-time encoding performed each time a speechsignal is received partially during the user utterance, the intentclassifier 220 may determine the intent information of the speech signalfurther based on type information of the one or more segments determinedby the segment classifier 230. In summary, type information of segments,determined by the segment classifier 230 based on linguisticcharacteristics, is transmitted to and utilized by the intent classifier220.

In another example of obtaining context information by encoding theentire speech signal in response to completion of the user utterance,type information of segments, determined by the segment classifier 230irrespective or independent of linguistic characteristics of theutterance, may be transmitted to and utilized by the intent classifier220.

In some examples, the segment classifier 230 may determine the typeinformation of the segment included in the speech signal based on theintent information determined by the intent classifier 220, in additionto the context information from the encoder 210. For example, inresponse to the intent classifier 220 determining intent informationincluding “sending a message,” the type information of the segmentscorresponding to a recipient and message content for sending a messageis determined relatively quickly with relatively high accuracy furtherbased on such intent information.

In a language where a verb is typically placed in the front portion of asentence, like English, a user's intent may be included or representedat the front portion of an utterance. Thus, in an example of obtainingcontext information through real-time encoding performed each time aspeech signal is partially received during the user utterance, thesegment classifier 230 may additionally utilize the intent informationof the speech signal determined by the intent classifier 220. Insummary, the intent information of the speech signal determined by theintent classifier 220 based on linguistic characteristics may betransmitted to and utilized by the segment classifier 230.

In another example of obtaining context information by encoding theentire speech signal in response to completion of the user utterance,the intent information of the speech signal determined by the intentclassifier 220 irrespective of linguistic characteristics may betransmitted to and utilized by the segment classifier 230.

The total number of speech recognition decoders implemented, e.g.,including speech recognition decoders 241 and 242, may be equal to atotal number of segments identified by the segment classifier 230, whereeach implemented decoder may output a segment value of a correspondingsegment by performing recognition with respect to the segment. Each ofthe implemented speech recognition decoders may receive the contextinformation from the encoder 210, and may receive additional informationrelated to the corresponding segment from the segment classifier 230.Here, the additional information may include type information andattention information of the corresponding segment.

For example, the speech recognition decoder 241 may be a decodercorresponding to a segment with type information of “recipient.” Thespeech recognition decoder 241 may identify a portion of the contextinformation corresponding to the recipient segment based on theattention information and outputs “Peter” as a segment value of thesegment by decoding the identified portion. Similarly, the speechrecognition decoder 242 may be a decoder corresponding to a segment withtype information of “content.” The speech recognition decoder 242 mayidentify a portion of the context information corresponding to thecontent segment based on the attention information and outputs “have youhad lunch” as a segment value of the segment by decoding the identifiedportion. As described above, the speech recognition decoders 241 and 242each perform speech recognition with respect to only a singlecorresponding segment.

The speech recognition decoders 241 and 242 may determine segment valuesof the respective segments by decoding, in parallel, the portions of thecontext information identified as corresponding to the respectivesegments. In other words, the segment classifier 230 may identifyportions desiring speech recognition. The speech recognition decoders241 and 242 may perform speech recognition with respect to theidentified portions in parallel, such that the portions subject toactual speech recognition in the entire speech signal may be minimized.The time for speech recognition may be minimized through parallel speechrecognition with respect to the portions, which may improve therecognition accuracy over previous approaches. For example, theprocessing rate of the examples in this disclosure may increase whencompared to an example where natural language processing is performedafter speech recognition of the entire speech signal based on acascading model in which a separate speech recognition engine and anatural language understanding (NLU) engine are connected in series. Inaddition, in the examples of this disclosure, a reduction in the overallaccuracy caused by speech recognition errors being delivered to the NLUengine may be prevented.

In some examples, based on the type information of the segments, thespeech recognition decoders 241 and 242 may further perform speechrecognition with respect to the portions of the context informationcorresponding to respective segments. For example, based on the typeinformation including “recipient”, the speech recognition decoder 241may further perform speech recognition with respect to a correspondingportion of the context information, such that a name of a person or acontact name stored in an electronic device is output.

Further, the speech recognition decoders 241 and 242 may perform speechrecognition with respect to the portions of the context informationcorresponding to the respective segments further based on the intentinformation of the speech signal. For example, based on the intentinformation, including “sending a message,” the speech recognitiondecoder 241 may improve the accuracy of a speech recognition withrespect to the corresponding portion of the context information.

In FIG. 2, depending on the example, connections indicated by brokenlines may be omitted.

Referring to FIG. 3, a speech processing apparatus includes a segmentdecoder 310 corresponding to a segment classifier provided in the formof a decoder. Attention information related to respective segments maybe obtained through the segment decoder 310.

The segment decoder 310 may sequentially identify one or more segmentsincluded in a speech signal based on context information from anencoder. For example, the segment decoder 310 may identify a firstsegment in the speech signal and then identify a subsequent segmentbased on a result of the first identification. In this manner, thesegments included in the speech signal are sequentially identified untila segment is no longer identified in the speech signal. The segmentdecoder 310 may be an autoregressive model.

In response to the segments included in the speech signal beingsequentially identified by the segment decoder 310, additionalinformation, for example, type information, attention information, andthe like, of the identified segments may be transmitted to speechrecognition decoders 320. The speech recognition decoders 320 may eachperform speech recognition with respect to a corresponding segment eachtime additional segments are received from the segment decoder 310.Thus, the speech recognition decoders 320 may also sequentially performspeech recognition with respect to the segments.

As described with reference to FIG. 3, when the segment decoder 310determines type information of the segments, intent information of thespeech signal determined by an intent classifier may be additionallyused. Further, when the intent classifier determines the intentinformation of the speech signal, the type information of the segmentsdetermined by the segment decoder 310 may be additionally used. Furtherstill, when the speech recognition decoders 320 perform speechrecognition with respect to the segments, the intent information of thespeech signal and/or the type information of the segments may beadditionally used.

In FIG. 3, connections indicated by broken lines may be omitteddepending on an example.

FIG. 4 illustrates an example of a speech processing method.

Referring to FIG. 4, a speech processing method performed by a processorincluded in a speech processing apparatus is illustrated.

In operation 410, the speech processing apparatus obtains contextinformation from a speech signal of a user using a neural network-basedencoder, for example. In response to a determination that the speechsignal includes a plurality of segments, the speech processing apparatusmay determine segment values of the segments by recognizing, in parallelusing a plurality of decoders, portions of the context informationidentified as corresponding to the respective segments based onattention information respectively corresponding to the plurality ofsegments.

In operation 420, the speech processing apparatus determines the intentinformation of the speech signal based on the context information. Thespeech processing apparatus may further determine the intent informationof the speech signal based on the type information of one or moresegments included in the speech signal.

In operation 430, the speech processing apparatus determines attentioninformation corresponding to a segment included in the speech signalbased on the context information. If a segment classifier is provided inthe form of a decoder, a plurality of segments included in the speechsignal may be sequentially identified by the segment classifier providedin the form of a decoder.

Further, the speech processing apparatus may determine the typeinformation of the segment included in the speech signal based on thecontext information. The speech processing apparatus may furtherdetermine the type information of the segment based on the intentinformation of the speech signal.

In operation 440, the speech processing apparatus determines a segmentvalue of the segment by recognizing, using a decoder, a portion of thecontext information identified as corresponding to the segment based onthe attention information.

The speech processing apparatus may further determine the segment valueof the segment by recognizing the identified portion based on at leastone of the type information of the segment and the intent information ofthe speech signal.

If the segment classifier is provided in the form of a decoder, thespeech processing apparatus may sequentially determine the segmentvalues of the respective segments in response to the segments beingsequentially identified by the segment classifier provided in the formof a decoder.

In operation 450, the speech processing apparatus performs an operationcorresponding to the intent information based on the segment value ofthe segment and the type information. In some examples, the operationcorresponding to the intent information may be performed by anelectronic device other than the speech processing apparatus.

The description provided with reference to FIGS. 1 through 3 applies tothe operations of FIG. 4, and thus duplicate description will be omittedfor conciseness.

FIG. 5 illustrates an example of a speech processing apparatus.

Referring to FIG. 5, a speech processing apparatus 500 includes a memory510, a processor 520, and an input/output (I/O) interface 530. Thememory 510, the processor 520, and the I/O interface 530 communicatewith each other through a bus 540.

The memory 510 stores computer-readable instructions. The processor 520,by one or more processors, performs the operations described above inresponse to the instructions stored in the memory 510 being executed bythe processor 520. The memory 510 is a volatile memory or a non-volatilememory.

The processor 520 is a device configured to execute the instructions orprograms or to control the speech processing apparatus 500, andincludes, for example, a central processing unit (CPU) and/or a graphicsprocessing unit (GPU). The speech processing apparatus 500 may beconnected to an external device, for example, an electronic device or amicrophone, through the I/O interface 530, and exchanges data with theexternal device.

The processor 520 may obtain context information from a speech signal ofa user using a neural network-based encoder, for example, determineintent information of the speech signal based on the contextinformation, determine attention information corresponding to a segmentincluded in the speech signal based on the context information, anddetermine a segment value of the segment by recognizing, using adecoder, a portion of the context information identified ascorresponding to the segment based on the attention information.

A neural network used by the speech processing apparatus 500 todetermine the intent information of the speech signal and the segmentmay be trained to determine ground-truth intent information and aground-truth segment, in response to a training speech signal beingprovided.

In addition, the speech processing apparatus 500 may perform any or allof the operations described above.

The speech processing apparatuses, the speech processing apparatuses 110and 500, and other apparatuses, units, modules, devices, and othercomponents described herein with respect to FIGS. 1-5 are implemented byhardware components. Examples of hardware components that may be used toperform the operations described in this application where appropriateinclude controllers, sensors, generators, drivers, memories,comparators, arithmetic logic units, adders, subtractors, multipliers,dividers, integrators, and any other electronic components configured toperform the operations described in this application. In other examples,one or more of the hardware components that perform the operationsdescribed in this application are implemented by computing hardware, forexample, by one or more processors or computers. A processor or computermay be implemented by one or more processing elements, such as an arrayof logic gates, a controller and an arithmetic logic unit, a digitalsignal processor, a microcomputer, a programmable logic controller, afield-programmable gate array, a programmable logic array, amicroprocessor, or any other device or combination of devices that isconfigured to respond to and execute instructions in a defined manner toachieve a desired result. In one example, a processor or computerincludes, or is connected to, one or more memories storing instructionsor software that are executed by the processor or computer. Hardwarecomponents implemented by a processor or computer may executeinstructions or software, such as an operating system (OS) and one ormore software applications that run on the OS, to perform the operationsdescribed in this application. The hardware components may also access,manipulate, process, create, and store data in response to execution ofthe instructions or software. For simplicity, the singular term“processor” or “computer” may be used in the description of the examplesdescribed in this application, but in other examples multiple processorsor computers may be used, or a processor or computer may includemultiple processing elements, or multiple types of processing elements,or both. For example, a single hardware component or two or morehardware components may be implemented by a single processor, or two ormore processors, or a processor and a controller. One or more hardwarecomponents may be implemented by one or more processors, or a processorand a controller, and one or more other hardware components may beimplemented by one or more other processors, or another processor andanother controller. One or more processors, or a processor and acontroller, may implement a single hardware component, or two or morehardware components. A hardware component may have any one or more ofdifferent processing configurations, examples of which include a singleprocessor, independent processors, parallel processors,single-instruction single-data (SISD) multiprocessing,single-instruction multiple-data (SIMD) multiprocessing,multiple-instruction single-data (MISD) multiprocessing, andmultiple-instruction multiple-data (MIMD) multiprocessing.

The methods illustrated in FIGS. 1-5 that perform the operationsdescribed in this application are performed by computing hardware, forexample, by one or more processors or computers, implemented asdescribed above executing instructions or software to perform theoperations described in this application that are performed by themethods. For example, a single operation or two or more operations maybe performed by a single processor, or two or more processors, or aprocessor and a controller. One or more operations may be performed byone or more processors, or a processor and a controller, and one or moreother operations may be performed by one or more other processors, oranother processor and another controller. One or more processors, or aprocessor and a controller, may perform a single operation, or two ormore operations.

Instructions or software to control computing hardware, for example, oneor more processors or computers, to implement the hardware componentsand perform the methods as described above may be written as computerprograms, code segments, instructions or any combination thereof, forindividually or collectively instructing or configuring the one or moreprocessors or computers to operate as a machine or special-purposecomputer to perform the operations that are performed by the hardwarecomponents and the methods as described above. In one example, theinstructions or software include machine code that is directly executedby the one or more processors or computers, such as machine codeproduced by a compiler. In another example, the instructions or softwareincludes higher-level code that is executed by the one or moreprocessors or computer using an interpreter. The instructions orsoftware may be written using any programming language based on theblock diagrams and the flow charts illustrated in the drawings and thecorresponding descriptions in the specification, which disclosealgorithms for performing the operations that are performed by thehardware components and the methods as described above.

The instructions or software to control a processor or computer toimplement the hardware components and perform the methods as describedabove, and any associated data, data files, and data structures, arerecorded, stored, or fixed in or on one or more non-transitorycomputer-readable storage media. Examples of a non-transitorycomputer-readable storage medium include read-only memory (ROM),random-access programmable read only memory (PROM), electricallyerasable programmable read-only memory (EEPROM), random-access memory(RAM), dynamic random access memory (DRAM), static random access memory(SRAM), flash memory, non-volatile memory, CD-ROMs, CD−Rs, CD+Rs,CD−RWs, CD+RWs, DVD-ROMs, DVD−Rs, DVD+Rs, DVD−RWs, DVD+RWs, DVD-RAMs,BD-ROMs, BD-Rs, BD-R LTHs, BD-REs, blue-ray or optical disk storage,hard disk drive (HDD), solid state drive (SSD), flash memory, a cardtype memory such as multimedia card micro or a card (for example, securedigital (SD) or extreme digital (XD)), magnetic tapes, floppy disks,magneto-optical data storage devices, optical data storage devices, harddisks, solid-state disks, and any other device that is configured tostore the instructions or software and any associated data, data files,and data structures in a non-transitory manner and providing theinstructions or software and any associated data, data files, and datastructures to a processor or computer so that the processor or computercan execute the instructions.

While this disclosure includes specific examples, it will be apparentafter an understanding of the disclosure of this application thatvarious changes in form and details may be made in these exampleswithout departing from the spirit and scope of the claims and theirequivalents. The examples described herein are to be considered in adescriptive sense only, and not for purposes of limitation. Descriptionsof features or aspects in each example are to be considered as beingapplicable to similar features or aspects in other examples. Suitableresults may be achieved if the described techniques are performed in adifferent order, and/or if components in a described system,architecture, device, or circuit are combined in a different manner,and/or replaced or supplemented by other components or theirequivalents. Therefore, the scope of the disclosure is defined not bythe detailed description, but by the claims and their equivalents, andall variations within the scope of the claims and their equivalents areto be construed as being included in the disclosure.

What is claimed is:
 1. A processor-implemented speech processing method,comprising: obtaining context information from a speech signal of a userusing a neural network-based encoder; determining intent information ofthe speech signal based on the context information; determining, basedon the context information, attention information corresponding to asegment included in the speech signal; and determining, based on theattention information, a segment value of the segment by recognizing,using a decoder, a portion of the context information identified ascorresponding to the segment.
 2. The speech processing method of claim1, wherein in response to a determination that the speech signalincludes a plurality of segments, the determining of the segment valueof the segment comprises determining segment values of each of thesegment by recognizing, in parallel using a plurality of decoders,portions of the context information identified as corresponding torespective segment based on attention information of the respectivesegment.
 3. The speech processing method of claim 1, further comprising:determining type information of the segment included in the speechsignal based on the context information, wherein the determining of thesegment value of the segment further comprises recognizing the portionbased on the type information of the segment.
 4. The speech processingmethod of claim 1, wherein the determining of the segment value of thesegment further comprises recognizing the portion based on the intentinformation of the speech signal.
 5. The speech processing method ofclaim 1, wherein type information of the segment is determined based onthe context information and the intent information of the speech signal.6. The speech processing method of claim 1, wherein the intentinformation is further determined based on type information of one ormore segments included in the speech signal.
 7. The speech processingmethod of claim 1, wherein respective segments included in the speechsignal are sequentially identified by a segment classifier provided in aform of a decoder.
 8. The speech processing method of claim 7, whereinthe determining of the segment value of the segment comprisessequentially determining segment values of the respective segments inresponse to the respective segments being sequentially identified by thesegment classifier provided in the form of a decoder.
 9. The speechprocessing method of claim 1, further comprising: performing anoperation corresponding to the intent information based on the segmentvalue of the segment and type information.
 10. A non-transitorycomputer-readable storage medium storing instructions that, whenexecuted by a processor, cause the processor to perform the speechprocessing method of claim
 1. 11. A speech processing apparatus,comprising: one or more processors configured to: obtain contextinformation from a speech signal of a user using a neural network-basedencoder; determine intent information of the speech signal based on thecontext information; determine, based on the context information,attention information corresponding to a segment included in the speechsignal; and determine, based on the attention information, a segmentvalue of the segment by recognizing, using a decoder, a portion of thecontext information identified as corresponding to the segment.
 12. Thespeech processing apparatus of claim 11, wherein the one or moreprocessors are further configured to: determine whether the speechsignal includes a plurality of segments, and determine segment value ofeach of the segment by recognizing, in parallel using a plurality ofdecoders, portions of the context information identified ascorresponding to a respective segment based on respective attentioninformation of the respective segment.
 13. The speech processingapparatus of claim 11, wherein the one or more processors are furtherconfigured to: determine type information of the segment included in thespeech signal based on the context information, and determine, based onthe type information of the segment, the segment value of the segment byrecognizing the portion.
 14. The speech processing apparatus of claim11, wherein the one or more processors are further configured todetermine, based on the intent information of the speech signal, thesegment value of the segment by recognizing the portion.
 15. The speechprocessing apparatus of claim 11, wherein type information of thesegment is determined based on the context information and the intentinformation of the speech signal.
 16. The speech processing apparatus ofclaim 11, wherein the one or more processors are further configured todetermine the intent information of the speech signal based on typeinformation of one or more segments included in the speech signal. 17.The speech processing apparatus of claim 11, wherein respective segmentsincluded in the speech signal are sequentially identified by a segmentclassifier provided in a form of a decoder.
 18. The speech processingapparatus of claim 17, wherein the one or more processors are furtherconfigured to sequentially determine segment values of the respectivesegments in response to the segments being sequentially identified bythe segment classifier provided in the form of a decoder.
 19. The speechprocessing apparatus of claim 11, wherein the one or more processors arefurther configured to perform an operation corresponding to the intentinformation based on the segment value of the segment and typeinformation.
 20. The speech processing apparatus of claim 11, furthercomprising: a memory storing instructions that when executed by the oneor more processors configures the one or more processors to perform theobtaining of the context information, the determining of the intentinformation, the determining of the attention information, and thedetermining of the segment value of the segment.